Installing an AVED-enabled Condor node
RPM installation
- As root install condor RPM
rpm -ivh condor-xxxxx-linux-x86-rhel3-dynamic-1.i386.rpm
error: Failed dependencies: libstdc++.so.5 is needed by condor-xxxxxxxxx
Install libstdc+.so.5: yum -y install compact-libstdc+-33 - Condor require the CONDOR_CONFIG variable sets to the emplacement of its config file
export CONDOR_CONFIG=/opt/condor-xxxxxxxxx/etc/condor_config
- Edit /opt/condor-xxxxxx/etc/condor_config changing the lines to allow READ/WRITE access and an address where email should be sent when something goes wrong
HOSTALLOW_WRITE = *.your.domain CONDOR_ADMIN = <your email here>
- Run configure script to setup installation as a submit and execute only node
/opt/condor-xxxxxx/condor_configure --type=submit,execute,manager --owner=user --install-dir=/opt/condor-xxxxx
Make sure in /opt/condor-xxxxxx/etc/condor_config line reads: DAEMON_LIST = COLLECTOR, MASTER, NEGOTIATOR, SCHEDD, STARTD
Installing Condor as a service on Linux
- Create condor.sh file in /etc/profile.d then add the following to it:
CONDOR_ROOT=/opt/condor-xxxxxxxxxxx export PATH=$PATH:$CONDOR_ROOT/sbin:$CONDOR_ROOT/bin export CONDOR_CONFIG=$CONDOR_ROOT/etc/condor_config
- Install condor service script to execute condor on bootup
cd /opt/condor-xxxxxx/etc/examples cp condor.boot /etc/init.d/condor
- Edit condor service script replacing MASTER line
MASTER=$CONDOR_ROOT/sbin/condor_master
- Add . /etc/profile.d/condor.sh before the line MASTER
. /etc/profile.d/condor.sh MASTER=$CONDOR_ROOT/sbin/condor_master
- Add soft links to the condor startup script
ln -s /etc/init.d/condor /etc/rc2.d/S100condor ln -s /etc/init.d/condor /etc/rc2.d/K100condor
- Start the condor service
service condor start
Configuring Condor
Change the following value in the /opt/condor-xxxxxxx/local.xxxxxx/condor_config.local
## When is this machine willing to start a job? START = TRUE ## When to suspend a job? SUSPEND = FALSE ## When to nicely stop a job? ## (as opposed to killing it instantaneously) PREEMPT = FALSE ## When to instantaneously kill a preempting job ## (e.g. if a job is in the pre-empting stage for too long) KILL = FALSE
Installing AVED as a web-service
A web-service has been written to make the system easy to interface and automate. AVED web-service requires AVED and its dependencies to be install (cf. AVED Guide section Installation) and requires Condor as well (cf. AVED Guide section Special Features-Installing an AVED-enabled Condor node). Moreover, in order to deploy AVED web-service, a web application server is required. We'll consider in this Guide starting from scratch, so if you already have an application server runing and install go to section: Deploy the AVED web-service.
The AVED web-service is distributed as source files that you must compile yourself, however, a ant script is provided to help you in this step. Here is a short overview of the steps to build and deploy the AVED web -service from sources:
- Preparation: AVED web-services required a web server and ant to be built. See Installing AVED as a web-service - Preparation section for more information.
- Download the AVED source files (distributed as tar'ed and compressed files). Place the file in a place of your own choice.
- Untar and uncompressthe file to the place you want the program installed. You will then need to read the next section about how to configure before you compile the program. Below is shown the exact commands: tar -xzvf /path/to/aved-web-service-1.0.0.tar.gz
- Now change to the new directory
cd aved-webservice
- Run ant.
ant
- copy and paste the build/AvedWebService.aar files into the <path-to-the-web-server>/webapps/axis2/WEB-INF/services
cp build/AvedWebService.aar <path-to-the-web-server>/webapps/axis2/WEB-INF/services
- copy and paste the jars needed by the web service from the jars folder into the <path-to-the-web-server>/webapps/axis2/WEB-INF/lib
cp jars/*.jar <path-to-the-web-server>/webapps/axis2/WEB-INF/lib - restart your web server application
Preparation for deploy the AVED web-service
In this section we're gonna install the application server apache Tomcat, the web-service utils with apach Axis2, and finally ant to be able to built the AVED web-service. If you already have an application web-server install and runing you can directly go to the Deploy the AVED web-service.
Installing Java
Apache Tomcat and ant require Java to be install on your computer. You can process different way: RPMs, self-extracted, or sources. If you're using sources, download the binaries distribution from http://www.sun.com/java/. Then untat and set the JAVA_HOME environmental variable to the full path of your Java distribution.
tar xzvf jdkx.xxx.tar.gz export JAVA_HOME=<full-path-to-Java>
Installing apache Tomcat
To install Apache tomcat, first download it from http://tomcat.apache.org/ (download the core version). Then extract it into the folder where you want to setup your application server.
tar xzvf apache-tomcat-xxxxxx.tar.gz
To test if your application server is on, just start it, you have to be in the apache-tomcat folder:
./bin/catalina.sh start
Start your favorite browser and tape: http://localhost:8080/, if everything is working you should fall on the apache-tomcat presentation page. Now your application server is up, download the web-service utilities from http://ws.apache.org/axis2/ (download the war distribution) and copy it into the <full-path-to-tomcat>/webapps folder.
cp <path-to-axis2>/axis2.war <path-to-tomcat>/webapps
Then on your browser, tape: http://localhost:8080/axis2. You should find at this URL the axis2 presentation page.
Installing apache Ant
The binarie distribution can be found at: http://ant.apache.org/bindownload.cgi. Download it and untar it into the folder you want ant to be install. You can install it from rpms as well. Ant is not required to build the AVED web-service, it's just more convinient to build it, but you can compile it by yourself using javac. Don't forget to add to the classpath the jars included into the aved-webservice/jars folder.
tar xzvf apache-antxxxxxxx.tar.gz
Allowing condor to be requested from http
Open the file <path-to-condor>/etc/condor_config and add the following lines (wherever in part 1)
SCHEDD_ARGS=-p 8181 ENABLE_SOAP = TRUE ALLOW_SOAP = */* ENABLE_WEB_SERVER = TRUE QUEUE_ALL_USERS_TRUSTED = TRUE
You can set the SCHEDD_ARGS to the port you prefere, here we choose 8181. You have to restart condor or juste reconfig it:
condor_reconfig
Deploy the AVED web-service
To deploy the AVED web-service, ant and an application web-server are required, if you don't have any please refer to AVED Guide - Special Features section Installing AVED as a web-service - Preparation for deploy the AVED web-service.
The web service provided condains the service (src/org/mbari/aved/webservice/server/AvedService.java) and an exemple to request it (src/org/mbari/aved/webservice/client/AvedClient.java). Before deploying the web-service, you have to edit the java code in order to provide the user Condor is gonna run with and the computer sets. For that edit the file AvedService.java in src/org/mbari/aved/webservice/server
public static String SCHEDD_LOCATION = "http://host.domain.name.org:8181"; public static String USER = "aved";
Then, compile the source by tapping
ant
If everything is going well, you should see a BUILD SUCCESSFULL printed, and a build folder should have been created. In these folder you'll find a AvedService.aar file what is the web-service ! To deploy it, just copy and past this file into your application server directory, and the jars from the jars into the application server lib directory:
cp build/AvedService.aar <full-path-to-tomcat>/webapps/axis2/WEB-INF/services/ cp jars/*.jar <full-path-to-tomcat>/webapps/axis2/WEB-INF/lib/
Restart tomcat, and check with your browser if your web-service is up by tapping http://host:8080/axis2/services/listServices, AvedService should be listed.
The AVED web-service requires 4 envirenmental variables:
export MBARIVISIONROOT=<full-path-to-mbarivision> export JAVA_HOME=<full-path-to-Java> export CONDOR_CONFIG=/opt/condor-xxxxxxxxx/etc/condor_config export CONDOR_SPOOL=/opt/condor-xxxxxxx/local.xxxxxxx/spool
You can know test it with the client code provided !
The web service is composed of 3 functions:
- OMElement sendJobToAved (String path): this function takes for parameter the path of the video to be processed and return an XML file with node1=the cluster number and node2=the job number
- OMElement getJobStatus (int cluster, int job): this function takes the cluster number and the job number and return the job status (1: Idle, 2:Running, 3:Removed, 4:Completed, 5:Held).
- OMElement getResult (int cluster, int job): this function takes the cluster number and the job number and return the XML result of the processing.
The web service client is basicly a java code what send a job to AVED wait until the status id is at 4 (completed) and get the result.
Installing the AVED web-service as a service on Linux
- Create tomcat.sh file in /etc/profile.d then add the following to it:
export JAVA_HOME=<full-path -to-jdk> export MBARIVISIONROOT=<full-path-to-mbarivision> export CONDOR_SPOOL=/opt/condor-xxxxxxx/local.xxxxxxx/spool
- Create tomcat file in /etc/init.d then add the following to it:
. /etc/profile.d/tomcat.sh <full-path-to-tomcat>/bin/catalina.sh start
* Add soft links to the tomcat startup script
ln -s /etc/init.d/tomcat /etc/rc2.d/S100tomcat ln -s /etc/init.d/tomcat /etc/rc2.d/K100tomcat
Start the tomcat service
service tomcat start
